Wide dynamic range operation for CMOS sensor with freeze-frame shutter

ABSTRACT

Wide dynamic range operation is used to write a signal in a freeze-frame pixel into the memory twice, first after short integration and then after long integration. The wide dynamic range operation allows the intra-scene dynamic range of images to be extended by combining the image taken with a short exposure time with the image taken with a long exposure time. A freeze-frame pixel is based on voltage sharing between the photodetector PD and the analog memory. Thus, with wide dynamic range operation, the resulting voltage in the memory may be a linear superposition of the two signals representing a bright and a dark image after two operations of sampling.

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This invention claims priority under 35 U.S.C. 119/120 fromprovisional application serial No. 60/243,898 filed Oct. 26, 2000.

TECHNICAL FIELD

[0002] This invention relates to CMOS sensors, and more particularly tooperating with a wide dynamic range using freeze-frame shutters.

BACKGROUND

[0003] An active pixel sensor (“APS”) is a special kind of light sensingdevice. Each active pixel includes a light sensing element and one ormore active transistors within the pixel itself. The active transistorsamplify and buffer the signals generated by the light sensing elementsin the pixels. One type of such APS devices is disclosed in U.S. Pat.No. 5,471,515 by Fossum et al., the disclosure of which is incorporatedherein by reference.

[0004] There are many applications for active pixel image sensors,including scientific, as well as commercial and consumer applications.The special techniques of active pixel sensing allow using asemiconductor family formation process which is compatible with CMOS,e.g., NMOS. This technique enables the readout electronic to beintegrated on the wafer using a similar process. The result is a highperformance sensor with high quantum efficiency and low dark current.

[0005] CMOS active pixel image sensors may be operated using a “rolling”shutter. Such a shutter operates by reading out each row of pixels, andthen resetting that individual row, and then rolling to read and thenreset the next row of pixels. Each pixel hence gets read and then resetat slightly different times. Hence, each pixel has a slightly differenttime of integration. Some applications, such as high-speed photography,may require more time consistency than is possible using this approach.Therefore, in these other applications, a frame shutter may be used. Inthe frame shutter mode, all pixels in the array have substantiallyidentical integration start times and integration stop times.

[0006] A wide dynamic range (WiDyR) technique was developed for CMOSsensors with a rolling shutter and is described in U.S. Pat. No.6,115,065, which is incorporated herein by reference. The WiDyRtechnique allows the extension of the intra-scene dynamic range ofimages by combining an image taken with a short exposure time with animage taken with a long exposure time. U.S. Pat. No. 6,115,065 teachesdesigns and operational methods to increase the dynamic range of imagesensors and APS devices in particular by achieving more than oneintegration times for each pixel thereof. An APS system with more thanone column-parallel signal chains for readout are described formaintaining a high frame rate in readout. Each active pixel is sampledfor multiple times during a single frame readout, thus resulting inmultiple integration times. The operation methods can also be used toobtain multiple integration times for each pixel with an APS designhaving a single column-parallel signal chain for readout. Furthermore,analog-to-digital conversion of high speed and high resolution can beimplemented.

SUMMARY

[0007] Wide dynamic range operation is used to write a signal in afreeze-frame pixel into the memory twice, first after short integrationand then after long integration. The wide dynamic range operation allowsthe intra-scene dynamic range of images to be extended by combining theimage taken with a short exposure time with the image taken with a longexposure time. A freeze-frame pixel is based on voltage sharing betweenthe photodetector PD and the analog memory. Thus, with wide dynamicrange operation, the resulting voltage in the memory may be a linearsuperposition of the two signals representing a bright and a dark imageafter two operations of sampling.

DESCRIPTION OF DRAWINGS

[0008] These and other features and advantages of the invention willbecome more apparent upon reading the following detailed description andupon reference to the accompanying drawings.

[0009]FIG. 1 illustrates one example of a freeze-frame pixel that may beused to obtain samples according to the present invention.

[0010]FIG. 2 illustrates sampling without using the wide dynamic rangeprocess according to one embodiment of the present invention.

[0011]FIG. 3 illustrates sampling using the wide dynamic range processaccording to one embodiment of the present invention.

[0012]FIG. 4 illustrates an idealized transfer curve which is achievableusing the present invention.

DETAILED DESCRIPTION

[0013] A freeze-frame pixel 100 is shown in FIG. 1 and comprises aphotodetector PD with capacitance C1, an analog memory C2, a sourcefollower transistor SF, switches S1-S3, and a row select switch RowSel.The photodetector PD with capacitance C1 is connected to a reset voltageV_(rst) through the switch S1. The analog memory C2 is connected to thereset voltage V_(rst) through switch S3. The photodetector PD withcapacitance C1 is connected to the analog memory C2 through the switchS2. The pixel structure 100 allows simultaneous photo-detection andreadout of data stored in the pixel memory during the previous frame.

[0014] The typical sequence of operations (without WiDyR) functions 200is illustrated graphically in FIG. 2. The switches S1 and S2 are enabledby active low pulses 205, 210, respectively. The analog memory C2 isreset during a previous readout time through switch S3. To start anexposure 215, the photodetector PD is reset through S1 at the first lowpulse in pulse train 205. To complete the integration 220 and transferthe data to the memory, the switch S2 is closed at the first low pulsein pulse train 210, thereby connecting the photodetector PD to theanalog memory C2. The readout of the data from the pixel is done row byrow. After the pixel signal is read 225, the analog memory is resetthrough S3. The reset level is also read out to subtract the pixelsource follower offset voltage.

[0015] The sequence of operations with the wide dynamic range process300 is illustrated graphically in FIG. 3. The switches S1 and S2 areenabled by active low pulses 305, 310, respectively. In wide dynamicrange operation, the idea is to write the signal into the memory twice,first after short integration and then after long integration. The pixel100 in FIG. 1 is based on voltage sharing between the photodetector PDand the analog memory C2. Thus, after two operations of sampling, theresulted voltage in the memory will be a linear superposition of the twosignals representing bright and dark image.

[0016] The wide dynamic range operation process 300 is performed byresetting the analog memory capacitor C2 to V_(rst) through the switchS3. The analog memory capacitor C2 may also be reset during the previousreadout. The process 300 then resets the photodetector PD to V_(rst)through the switch S1. The photodetector PD then begins integration of asignal to create a photocharge Q1 at time 315. After a short integrationperiod t1, the signal is sampled to the analog memory capacitor C2 byenabling the switch S2 for a short time. The photodetector PD continuesto integrate the signal for a long integration period t2 to create aphotocharge Q2. Following the long integration period t2, the signal isagain sampled to the analog memory capacitor C2 by enabling the switchS2. The signal is then read out 330 to all memories in the entire pixelarray.

[0017] Because of the capacitor divider effect, after two samplings fromthe photodetector PD, the resulted signal voltage will be a combinationof the signal accumulated during the short integration period t1 and thelong integration period t2. For corresponding photocharges Q1 and Q2 atthe photodetector PD, respectively, the signal voltage will be equal to:

V _(signal) =Q1 *C2/(C1+C2)² +Q2(C1+C2).

[0018] The signal voltage V_(signal) is a weighed sum of thephotocharges Q1 and Q2, so that the short integration is given a weightlesser by amount C2/(C1+C2).

[0019] The option of having wide dynamic range is useful when a portionof the image after the long exposure is saturated. When the longexposure is saturated, the resulting response after summing the twosignals will be the following:

V _(signal) =Q1 *C2/(C1+C2)² +Q _(sat)(C1+C2).

[0020] where Q_(sat) is the saturation charge for the photodetector PD.

[0021] Because the memories hold the short integration signal, theoverlap of the long integration and the frame readout are no longerpossible. However, this is not a problem for a pixel with a synchronousshutter. Typically the shutter is needed for making very short snaps, atleast 10 times shorter than the frame time. Then the relative increaseof frame time including the exposure will not be substantially lowerthan the frame rate of the image sensor.

[0022] The graph 400 in FIG. 4 illustrates the idealized transfer curvewhich is achievable using the present invention. The resulting slope forthe short integration is different from the original short exposureslope because of the C2/(C1+C2) factor.

[0023] However, one of the problems with the current pixel is that thesaturation voltage for photodiodes, approximately 0 V, is different fromthe pixel saturation voltage (−0.7 V for nMOS source follower SFtransistor). Also, the analog memory capacitance C2 is onlyapproximately ⅕ of the total capacitance of C1+C2. Therefore, the kneeof the combined transfer curve is below the threshold of the sourcefollower transistor SF and we do not see an improvement. Another problemcan arise from the bulk charge. If there is a noticeable charge left inthe pixel after saturation, then it can saturate the memory during thesecond sample and thus erase the signal stored in the analog memory C2after short integration.

[0024] These problems can be addressed to achieve the dual-slope widedynamic range response. First, the source follower SF and rowselectRowSel MOSFETS can be changed to p-MOS type transistors. Second, if thesignal-to-noise ration is not a problem, the size of the memorycapacitor C2 may be increased so that the knee of the combined transfercurve is above the threshold of the source follower transistor SF.Finally, using an antiblooming gate for the photodetector PD withpotential artifact such as fixed pattern noise due to antibloomingthreshold variation.

[0025] The present invention may be used with other pixels that do notuse the freeze-frame structures, such as a photogate, with chargetransfer rather that voltage sharing. The capacity of the detector atsaturation should not then exceed the capacity of the memory to avoiderasing of the signal kept in the analog memory C2 after shortintegration.

[0026] Numerous variations and modifications of the invention willbecome readily apparent to those skilled in the art. Accordingly, theinvention may be embodied in other specific forms without departing fromits spirit or essential characteristics.

What is claimed is:
 1. A method of obtaining an image using a CMOSsensor with a freeze-frame shutter comprising: collecting a short imagesignal during a first time period; sampling the short image signal afterthe first time period; collecting a long image signal during a secondtime period; sampling the long image signal after the second timeperiod; and combining the short image signal and the long image signalto create a total image signal.
 2. The method of claim 1, wherein thesecond time period includes the first time period.
 3. The method ofclaim 1, further comprising resetting a photodetector prior tocollecting the short image signal.
 4. The method of claim 1, furthercomprising resetting a memory containing the total image signal prior tocollecting the short image signal.
 5. The method of claim 1, furthercomprising simultaneous sampling of the short image signal whilecollecting the long image signal.
 6. The method of claim 1, furthercomprising reading the total image signal from the freeze-frame pixel.7. The method of claim 6, wherein the short image signals and the longimage signals are not collected during the reading of the total imagesignal.
 8. A freeze-frame pixel using wide dynamic range operatingcomprising: a photodetector having a memory; an analog memory; and aplurality of switches which connect the photodetector to the analogmemory, wherein a first switch allows collection of a first image signalby the photodetector, a second switch allows transfer of the first imagesignal from the photodetector memory to the analog memory while thephotodetector continues to collect a second image signal, and the secondswitch then allowing transfer of the second image signal to the analogmemory.
 9. The freeze-frame pixel of claim 8, wherein the first imagesignal is collected during a first time period and the second imagesignal is collected during a second time period, the second time periodbeing longer than the first time period.
 10. The freeze-frame pixel ofclaim 9, wherein the second time period includes the first time period.11. The freeze-frame pixel of claim 8, wherein a third switch allows thephotodetector and photodetector memory to be reset.
 12. The freeze-framepixel of claim 11, wherein a fourth switch allows the analog memory tobe reset.
 13. The freeze-frame pixel of claim 8, wherein the analogmemory combines the first image signal and the second image signal tocreate a total image signal.
 14. The freeze-frame pixel of claim 13,further comprising a readout section to transfer the total image signal.15. The freeze-frame pixel of claim 8, further comprising an array offreeze-frame pixels.